{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 9 - Using the GPU API\n",
    "\n",
    "**GOAL:** The goal of this exercise is to show how to use GPUs with remote functions and actors.\n",
    "\n",
    "**NOTE:** These exercises are designed to run on a machine without GPUs.\n",
    "\n",
    "See the documentation on using Ray with GPUs http://ray.readthedocs.io/en/latest/using-ray-with-gpus.html.\n",
    "\n",
    "### Concepts for this Exercise - Using Ray with GPUs\n",
    "\n",
    "We can indicate that a remote function or an actor requires some GPUs using the `num_gpus` keyword.\n",
    "\n",
    "```python\n",
    "@ray.remote(num_gpus=1)\n",
    "def f():\n",
    "    # The command ray.get_gpu_ids() returns a list of the indices\n",
    "    # of the GPUs that this task can use (e.g., [0] or [1]).\n",
    "    ray.get_gpu_ids()\n",
    "\n",
    "@ray.remote(num_gpus=2)\n",
    "class Foo(object):\n",
    "    def __init__(self):\n",
    "        # The command ray.get_gpu_ids() returns a list of the\n",
    "        # indices of the GPUs that this actor can use\n",
    "        # (e.g., [0, 1] or [3, 5]).\n",
    "        ray.get_gpu_ids()\n",
    "```\n",
    "\n",
    "Then inside of the actor constructor and methods, we can get the IDs of the GPUs allocated for that actor with `ray.get_gpu_ids()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import absolute_import\n",
    "from __future__ import division\n",
    "from __future__ import print_function\n",
    "\n",
    "import numpy as np\n",
    "import ray\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Start Ray, note that we pass in `num_gpus=4`. Ray will assume this machine has 4 GPUs (even if it does not). When a task or actor requests a GPU, it will be assigned a GPU ID from the set `[0, 1, 2, 3]`. It is then the responsibility of the task or actor to make sure that it only uses that specific GPU (e.g., by setting the `CUDA_VISIBLE_DEVICES` environment variable)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ray.init(num_cpus=4, num_gpus=2, include_webui=False, ignore_reinit_error=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**EXERCISE:** Change the remote function below to require one GPU.\n",
    "\n",
    "**NOTE:** This change does not make the remote function actually **use** the GPU, it simply **reserves** the GPU for use by the remote function. To actually use the GPU, the remote function would use a neural net library like TensorFlow or PyTorch after setting the `CUDA_VISIBLE_DEVICES` environment variable properly. This can be done as follows.\n",
    "\n",
    "```python\n",
    "import os\n",
    "os.environ['CUDA_VISIBLE_DEVICES'] = ','.join([str(i) for i in ray.get_gpu_ids()])\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@ray.remote\n",
    "def f():\n",
    "    time.sleep(0.5)\n",
    "    return ray.get_gpu_ids()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**VERIFY:** This code checks that each task was assigned one GPU and that not more than two tasks are run at the same time (because we told Ray there are only two GPUs)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "start_time = time.time()\n",
    "\n",
    "gpu_ids = ray.get([f.remote() for _ in range(3)])\n",
    "\n",
    "end_time = time.time()\n",
    "\n",
    "for i in range(len(gpu_ids)):\n",
    "    assert len(gpu_ids[i]) == 1\n",
    "\n",
    "assert end_time - start_time > 1\n",
    "\n",
    "print('Sucess! The test passed.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**EXERCISE:** The code below defines an actor. Make it require one GPU."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@ray.remote\n",
    "class Actor(object):\n",
    "    def __init__(self):\n",
    "        pass\n",
    "\n",
    "    def get_gpu_ids(self):\n",
    "        return ray.get_gpu_ids()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**VERIFY:** This code checks that the actor was assigned a GPU."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "actor = Actor.remote()\n",
    "\n",
    "gpu_ids = ray.get(actor.get_gpu_ids.remote())\n",
    "\n",
    "assert len(gpu_ids) == 1\n",
    "\n",
    "print('Sucess! The test passed.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
